fix: pd.to_numeric handling of datetime #62649
Open
+72
−4
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR handles all the cases of inconsistent datetime related
pd.to_numeric
handling from Issue 42380 and is compatible withpandas/io/parsers/base_parser.py
pandas/_libs/lib.pyx
contains the core/common logic for datetime relatedpd.to_numeric
handling, and is a singular place which handles scalar values,pd.Datetime
,pd.TimeStamp
andSeries(input_value).apply(partial(to_numeric))
pandas/pandas/_libs/lib.pyx
Line 2453 in 083f01a
pd.Timedelta
is returned as the direct value, andpd.Timestamp
with Timezones are handled with the default np.datetime64 UTC conversion.The removal of
elif lib.is_np_dtype(values_dtype, "mM"): values = values.view(np.int64)
inpandas/core/tools/numeric.py
addresses the bug where int value of NaT are returned (Issue 42380).Introduces
convert_datetime
flag inmaybe_convert_numeric
, as this function is used by bothpd.to_numeric
and the base_parsers, e.g., excel reader. The default behavior forpd.to_numeric
on datetimes is to convert to numeric, while the default behavior on base_parsers is to not convert.pandas/pandas/_libs/lib.pyi
Line 127 in 083f01a
pd.to_numeric
has an inconsistent behavior fordatetime
objects #43280pd.to_numeric
has an inconsistent behavior fordatetime
objects #43280 and related linked PR discussionsdoc/source/whatsnew/v3.0.0.rst
file